CSCI 2980 Project Report Data Migration from S-Store to BigDAWG

From spring 2016, I've been working with Prof. Stan Zdonik in a project about data migration from S-Store to BigDAWG polystore system. S-Store, which built on top of H-Store, is the world's first transactional streaming database system. S-Store maintains all the transactional support in a traditional relational database, while it supports streaming processing which is needed in the real-time applications. BigDAWG, which built on top of a variety of storage engines by MIT, is a polystore system provides cross-system querying, exploratory analysis, real-time decision supports and complex analytics. S-Store will serve as a front streaming engine within the BigDAWG, as well as a relational in-memory database in the relational island. In either case, the data within S-Store need to be migrated to different storage system in BigDAWG. This report first introduces these two systems, and then discusses the motivation and use cases for the migration process. Then it states the details about the decision and implementation, as well as the performance testing about the migrator implemented. In the last part of the report, I also proposed some observation and works that can be done in the future.